Machine Learning for Market Microstructure and High Frequency Trading

In this chapter, we overview the uses of machine learning for high frequency trading and market microstructure data and problems. Machine learning is a vibrant subfield of computer science that draws on models and methods from statistics, algorithms, computational complexity, artificial intelligence, control theory, and a variety of other disciplines. Its primary focus is on computationally and informationally efficient algorithms for inferring good predictive models from large data sets, and thus is a natural candidate for application to problems arising in HFT, both for trade execution and the generation of alpha. The inference of predictive models from historical data is obviously not new in quantitative finance; ubiquitous examples include coefficient estimation for the CAPM, Fama and French factors [5], and related approaches. The special challenges for machine learning presented by HFT generally arise from the very fine granularity of the data — often microstructure data at the resolution of individual orders, (partial) executions, hidden liquidity, and cancellations — and a lack of understanding of how such low-level data relates to actionable circumstances (such as profitably buying or selling shares, optimally executing a large order, etc.). In the language of machine learning, whereas models such as CAPM and its variants already prescribe what the relevant variables or “features” are for prediction or modeling (excess returns, book-to-market ratios, etc.), in many HFT problems one may have no prior intuitions about how (say) the distribution of liquidity in the order book relates to future price movements, if at all. Thus feature selection or feature engineering becomes an important process in machine learning for HFT, and is one of our central themes. Since HFT itself is a relatively recent phenomenon, there are few published works on the application of machine learning to HFT. For this reason, we structure the chapter around a few case studies from our own work [6, 14]. In each case study, we focus on a specific trading problem we would like to solve or optimize; the (microstructure) data from which we hope to solve this problem; the variables or features derived from the data as inputs to a machine learning process; and the machine learning algorithm applied to these features. The cases studies we will examine are: